Ranking for BioMinT: Investigating Performance, Local Search and Homonymy Recognition

In this paper we investigate several hypotheses concerning document relevance ranking for biological literature. More specifically, we focus on three topics: performance, risk of local searching, and homonymy recognition. Surprisingly, we find that a simple ranker based on the occurrence of a single word performs best. Adding this word as a new search term to each query yields even better results. The risk of local searching, i.e. retrieving documents by query followed by local indexing vs. maintaining a local repository and index of the full MEDLINE database, is found to be negligible – contrary to intuition, the latter approach even performs worse on some queries. The removal of automatically determined homonyms yields almost indistinguishable results to the original query, so it is not inconceivable that the problem of homonymy in biological literature has been overstated. Concluding, our work shows that simple rankers may be competitive to vastly more complex ensemble systems which include biological background knowledge and part-of-speech tagging which have been previously applied to task. The result reconfirms common wisdom within this research field.